System and method for dynamic stereoscopic calibration

ABSTRACT

Methods for stereo calibration of a dual-camera that includes a first camera and a second camera and system for performing such methods. In some embodiments, a method comprises obtaining optimized extrinsic and intrinsic parameters using initial intrinsic parameters and, optionally, initial extrinsic parameters of the cameras, estimating an infinity offset e using the optimized extrinsic and extrinsic parameters, and estimating a scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset parameter e, wherein the optimized extrinsic and extrinsic parameters, infinity offset e and scaling factor s are used together to provide stereo calibration that leads to improved depth estimation.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a 371 application from international patentapplication PCT/IB2020/051948 filed Mar. 6, 2020, and is related to andclaims priority from U.S. Provisional Patent Application No. 62/816,097filed on Mar. 9, 2019, which is expressly incorporated herein byreference in its entirety.

FIELD

Embodiments disclosed herein relate in general to optical instrumentcalibration such as in stereoscopic digital cameras, and moreparticularly to stereoscopic calibration in dual-aperture digitalcameras (“dual-cameras”) that are configured to be incorporated invehicles as part of a driver assistance system.

BACKGROUND Definitions

“Dynamic stereoscopic calibration”—estimation of “stereo parameters” ofa stereo (dual) camera without a known calibration chart, while thestereo camera is in use, with or without it being moved.

“Stereo parameters”: stereo camera parameters that are required to becalibrated in order to produce a high precision depth map, comprisingintrinsic parameters (for each camera) and extrinsic parameters (foreach camera pair).

“Intrinsic parameters”: parameters that include focal length, opticalaxis in X and Y axes and lens distortion coefficients.

“Extrinsic parameters”: parameters that include three relative (betweentwo cameras) angles (Yaw, Pitch and Roll) and three offsets (Tx, Ty andTz).

“Disparity axis”: the axis of disparity (in our XYZ coordinate system, Xif cameras are placed horizontally).

“Non-disparity axis”: the axis perpendicular to the disparity axis (inour XYZ coordinate system, Y if cameras are placed horizontally).

Advanced Driver-Assistance Systems (ADASs) are known. An ADAS includedin vehicles combine sensors and algorithms to understand the vehicle'senvironment so that a driver of the vehicle can receive assistance or bewarned of hazards. ADASs rely on computer vision, which plays a pivotalrole in acquiring, processing, analyzing, and understanding theenvironment and surrounding objects. Often, ADASs use multi-camerasystems with two or more cameras or “camera modules”.

FIG. 1A shows top views of vehicles having multi-camera systems invarious arrangements. The term “vehicle” may apply to any vehicleincluding (but not limited) to a car, a motorcycle, a truck, a bus, anairplane, a bicycle, etc. In (a) a dual-camera system includes twocameras 102 and 104 arranged close to each other along the X axis. Here,X the disparity axis is X and the a non-disparity axis is Y. The twocameras may share a common housing. In (b) two cameras 106 and 108 areplaced along the Y axis at a greater distance from each other than in(a). In (c), two cameras 110 and 112 are arranged along the X axis asdisparity axis. In (d) there are four cameras 114, 116, 118 and 120, in(e) there are three cameras 122, 124 and 126 and in (f) there are twocameras 128 and 130, arranged as shown. The cameras are not limited to aparticular type of camera. In an example, the cameras may be identical.In an example, cameras may differ in one or more of the followingparameters: focal length, sensor size, pixel size and/or pitch, and/orf-number (f/#). In an example, a camera may be a color camera, a blackand white camera or an infrared (IR) sensitive camera. In an example, amulti-camera system may additionally include infrared projectors, fiberoptics, lasers, sensors, or a combination thereof (not shown).

Accurate depth maps of the environment are necessary for the computervision to operate properly. A depth map is an image or image channelthat contains information relating to the distance of surfaces of sceneobjects from a viewpoint.

A common solution for creating depth maps of the environment is using astereoscopic camera or a dual-camera (a camera comprised of twosub-cameras) for imaging and estimating the distance of objects from thecamera. Using a dual-camera for depth map creation depends oncalculating disparity of the pixels of various objects in the field ofview (FOV). In order to accurately translate disparity values in pixelsto real-world depth in meters, there is a need for accurate camerastereo calibration.

Calibration of a stereo (or dual) camera system includes analyzingacquired data to assess the accuracy of the intrinsic and extrinsicparameters and adjusting accordingly.

Assuming all intrinsic and extrinsic parameters are known, an object'sdistance (or “depth) Z within the FOV of a vehicle dual-camera systemcan be calculated and/or estimated using equation 1:

$\begin{matrix}{Z = \frac{f*B}{D*{ps}}} & (1)\end{matrix}$where f is focal length, B is baseline, D is disparity in pixels, and“ps” is pixel size.

However, in practice, factory and dynamic calibration procedures sufferfrom estimation errors in intrinsic and/or extrinsic parameters that canbe expressed in a revised equation 1′:

$\begin{matrix}{Z = \frac{s*f*B}{\left( {D + e} \right)*{ps}}} & \left( 1^{\prime} \right)\end{matrix}$where “s” is an unknown scaling factor, an accumulative error in focallength estimation and translation along a disparity axis (i.e. Tx), and“e” represents a “infinity disparity error” or “infinity offset” thatencapsulates the estimation error in optical axis location for both left(“L) and right (“R”) cameras (intrinsic parameters) as well asestimation errors in the rotation along the “non-disparity axis”,extrinsic parameters).

FIG. 1B shows an example of the possible discrepancies in depthestimation for two exemplary errors in e. Assume e is estimated with 0.5pixels error or 1 pixels error. The figure shows a graph for a stereosystem with f=6 mm, ps=0.0042 mm and B=120 mm (see Eq. 1′). The graphdepicts diverging error percentages based on distance comparing a0.5-pixel error to a 1-pixel error (“e” in equation 1′). The effect ofeven half a pixel error on the depth estimation is dramatic especiallyin high distances.

Manual stereo calibrations before installation of stereo or dual-camerasin a host vehicle are difficult. Maintaining a pre-installation stereocalibration is difficult, due to changes during camera lifecycle. Suchchanges may include (but are not limited to) heat expansions, vibrationsand mechanical hits, which cause some of the calibration parameters tochange over time. Calibrating a stereo (or dual) camera mounted behind awindshield is further complicated since the windshield may affect someof the calibration parameters of the stereo camera, e.g. by distortingthe perspective or viewing angles of the camera. Therefore, thecalibration may be performed only after installing the cameras in thehost vehicle.

There have been a number of attempts to solve the stereoscopic cameracalibration issue, however, none have been able to devise a solutionthat meets the needs of industry. Some of these solutions attempt to runstructure from motion (SFM) algorithms. SFM uses complicated algorithmsthat track moving features in successive images to determine itsstructural information, then the image frames are processed to compute adepth map. This solution fails to meet the needs of industry becauserunning these processes is inordinately difficult and computationallydemanding for the cameras mounted in a moving car.

There is therefore a need for, and it would be advantageous to havedynamic stereoscopic calibration systems and methods that overcome thedeficiencies in existing systems and methods that use SFM technology.

SUMMARY

In various embodiments, there are provided methods for dynamicstereoscopic calibration of a stereo digital camera including a firstcamera and a second camera, each camera having intrinsic parameters andextrinsic parameters, the method comprising: obtaining optimizedextrinsic and intrinsic parameters based on input intrinsic parameters,and, optionally, input extrinsic parameters; estimating an offsetparameter e using the optimized extrinsic and extrinsic parameters;estimating a scaling factor s using the optimized extrinsic andextrinsic parameters and estimated offset parameter e; and using theoptimized extrinsic and extrinsic parameters, infinity offset e andscaling factor s to provide stereo calibration that leads to improveddepth estimation.

In certain embodiments, a method for dynamic stereoscopic calibrationdisclosed herein may include selecting initial values for the intrinsicand/or extrinsic parameters of the first camera and initial values forthe intrinsic parameters of the second camera. The initial values may bederived for example from the design of the camera (“nominal” values),from factory settings if calibration for each camera was done or fromprevious usage of the camera, etc. The calibration of the intrinsicand/or extrinsic parameters may include capturing at least one imagefrom the first camera and at least one image from the second camera,matching corresponding points on the at least one image from the firstcamera to corresponding points on the at least one image from the secondcamera, and calculating optimized intrinsic and extrinsic parameters ofthe first camera and the second camera using epipolar geometry. Thisprovides an initial calibration of the first camera and of the secondcamera with aligned epipolar lines. The various selections,calculations, processes etc. may be performed using a processor, anddata/results of the processing may be stored in a memory.

Further actions to estimate offset parameter e and scaling factor s mayinclude obtaining, at least two image pairs based upon images receivedfrom the first camera and the second camera, wherein the at least twoimage pairs are images sequentially taken via the first camera and thesecond camera, and wherein each pair of images (one from each camera)needs to be taken simultaneously; matching corresponding points on theat least two image pairs; and generating a disparity map, wherein thedisparity map includes pixels matched from the corresponding points onthe at least two image pairs, wherein pixels with constant disparity areidentified as pixels at infinity distance.

In certain embodiments, the method includes storing the at least twoimage pairs in a memory.

In certain embodiments, the number of at least two image pairs capturedfrom the first camera and the at least two image pairs captured from thesecond camera is determined by a processor.

In certain embodiments, the processor stops receiving at least two imagepairs from the first camera and the second camera once a full FOV iscaptured.

In certain embodiments, the stereo digital camera is installed in avehicle.

In certain embodiments, the stereo digital camera is configured to beincorporated in a vehicle as part of a driver assistance system.

In certain embodiments, the step of setting the initial intrinsicparameters of the first camera and the initial intrinsic parameters ofthe second camera includes a processor performing an initial guess forthe intrinsic parameters for said stereo digital camera.

In certain embodiments, the step of selecting initial intrinsicparameters include factory calibration.

In certain embodiments, the selecting initial intrinsic parametersincludes independent estimation from bundle adjustment.

In certain embodiments, selecting initial intrinsic parameters includesindependent estimation from structure from motion (SFM).

In certain embodiments, the at least one image from the first camera andat least one image from the second camera are stored in memory.

In certain embodiments, the corresponding points on the at least oneimage from the first camera and the second camera are stored in memory.

In certain embodiments, the disparity map is stored in memory.

In certain embodiments, the steps for calibrating external and internalparameters are repeated to obtain a full FOV.

In certain embodiments, the steps for calibrating depth are repeated toobtain a full FOV.

In certain embodiments, the intrinsic parameters are selected from agroup consisting of focal length, image distortion and optical axis.

In certain embodiments, the extrinsic parameters describe thetranslation and rotation of the one camera relative to the other.

In certain embodiments, the method includes using infinity disparity tocompensate for estimation errors.

In certain embodiments, the method includes identifying moving objectsin the at least two image pairs.

In certain embodiments, the method includes removing said moving objectsfrom the disparity map.

In certain embodiments, the moving objects are identified using computervision.

In certain embodiments, the moving objects are identified using highdisparity values.

In certain embodiments, the method includes repeating the steps of theabove referenced steps multiple times and averaging the results.

In an embodiment there is provided a method for dynamic stereo cameracalibration, comprising obtaining at least two image pairs from adual-camera, performing local registration of the at least two imagepairs and obtaining a registration map, finding a minimal disparity inthe registration map, calculating a minimum disparity value, defining aglobal minimal disparity value, and calibrating the dual-camera usingthe global minimal disparity value.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments herein described, by way of example only, withreference to the accompanying drawings. With specific reference now tothe drawings in detail, it is stressed that the particulars shown are byway of example and for purposes of illustrative discussion ofembodiments of the application. In this regard, the description takenwith the drawings makes apparent to those skilled in the art howembodiments of the application may be practiced. In the drawings:

FIG. 1A shows top views of vehicles having multi-camera systems invarious arrangements;

FIG. 1B shows an example of possible discrepancies in depth estimationdue to infinity disparity errors;

FIG. 2A shows a flowchart describing an exemplary embodiment forcalibrating stereo parameters;

FIG. 2B shows a flowchart describing an exemplary embodiment forcalibrating stereo parameters;

FIG. 3A shows a flowchart describing an exemplary embodiment forinfinity correction in a method disclosed herein;

FIG. 3B shows a flowchart describing an exemplary embodiment forinfinity correction in a method disclosed herein;

FIG. 4A shows a flowchart describing an exemplary embodiment of a methodfor estimating object scale in a method disclosed herein;

FIG. 4B shows a flowchart describing another exemplary embodiment of amethod for estimating object scale in a method disclosed herein;

FIG. 5 describes a system installed in a vehicle and used for performinga method disclosed herein.

DETAILED DESCRIPTION

Embodiments disclosed herein describe methods of dynamic stereocalibration of the intrinsic and extrinsic parameters that includeestimation of the “additional” parameters of equation 1′, scaling factors and disparity error e. While all the parameters in the equation may beestimated in a factory (where the dual-camera is assembled and/or wherethe vehicle is assembled), the parameters may change during the lifecycle of the camera due to a number of factors, including (but notlimited to) shift between sub-cameras, tilt between sub-cameras, shiftin each sub-camera between its lens and the sensor, change in the camerafocal length, etc.

The present application efficiently calibrates the camera parameters toensure that the camera is viewing its surroundings properly and is ableto effectively calculate distance.

FIG. 2A shows a flowchart describing an embodiment of an exemplarymethod for calibrating a stereo camera's intrinsic and extrinsicparameters. In step 202, initial extrinsic and/or intrinsic parametersare selected for both cameras. An initial calibration parameter can beestimated from factory settings (all the nominal values known for thestereo camera—focal length, lens distortion, optical axis, base line,etc.), through independent estimation from bundle adjustment, structurefrom motion, previous calibration parameters, etc. Intrinsic and(optionally) extrinsic parameters are calibrated in step 204. Thecalibration includes obtaining a first set of R-L images in sub-step 208and matching corresponding points in sub-step 210, using a featureextraction method on L and R images separately and finding thecorresponding feature of L image in the right image. A minimum of fourpairs of points are needed, but normally a few hundred pairs of pointsare used. The calibration continues with calculating extrinsicparameters in sub-step 212, using for example, Essential Metrixestimation and decomposition, estimating the 3 angles (Yaw, Pitch, Roll)and the translation up to unknown scale (three offsets) Tx, Ty, Tz. Theintrinsic parameters are then refined in sub-step 214 using anoptimization technique (i.e. gradient standard) to minimize thenon-disparity axis error over selected intrinsic parameters. Therefinement includes calculating the difference in image location foreach matched feature point along the non-disparity axis (NDA). The goalof the optimization is to minimize the sum of absolute NDA over allmatched features from all the images. In a perfectly calibrated stereosystem, the sum of absolute NDA will converge to zero. For practicalcases and for example, one can set a stop condition that the minimizedaverage absolute NDA be within a “delta” value from zero. For example,the delta value may be 0.1 pixels. Using another stop condition andexample, the stop condition may be a maximum absolute NDA smaller than0.25 pixels.

The output of step 204 is optimized stereo calibration (i.e. intrinsicand extrinsic) parameters 206, i.e. a calibrated dual-camera output thatallows rectifying the camera system's output into a pair of imageshaving parallel epipolar lines. The optimized stereo calibrationparameters are then used in estimating infinity offset e and scalingfactor s.

This optimization problem can be solved with a number of optimizationtechniques such as gradient decent. Intrinsic parameters refined in thissub-step include focal length ratio between left and right cameras, lensdistortion coefficients and “non-disparity” optical axis differences.

FIG. 2B shows a flowchart describing another embodiment of an exemplarymethod for calibrating a stereo camera's intrinsic and extrinsicparameters. This embodiment is similar to that of FIG. 2A, with thefollowing changes:

1. Iterate steps 208 and 210 until sufficient matched points aregathered: this is an iterative process performed in step 211 until thematched features are evenly spread across the camera's FOV and distancefrom the camera, for example by having 5 corresponding points in a 3Dbox of N⁰×N⁰×FOV_P_(disp) (N˜1/20 FOV P˜1/10 disparity range measured bypixels).

2. Iterate steps 212 and 214 until a stable state is reached: this is aniterative process performed in step 213. After intrinsic parameters wererefined in step 214 recalculate extrinsic parameters in step 212 andrefine until steady state is reached either in the parameter value or inthe sum of absolute NDA.

FIG. 3A shows a flowchart describing an exemplary embodiment of a methodfor infinity correction (i.e. for estimating infinity offset e inequation 1′). The method is implemented using a dual-camera in a dynamicenvironment (e.g. while driving in a given vehicle). At least two setsof stereo images (i.e. four images, 2L and 2R) are obtained while inmotion in step 302. Corresponding points in each set of L and R imagesare matched in step 304. Corresponding points between images in the twosets are matched in step 306. In contrast with the matching of left vs.right (L-R) features performed with a single set of images, in step 306the match is left vs. left (L-L) and/or right vs. right (R-R) in variouscar positions (i.e. of a same point in different sets of stereo imagesobtained in step 304). A disparity map of corresponding points isgenerated in step 308. The generation of the disparity map includescalculating the disparity value in the two time frames for all featuresmatched in both 304 and 306. This step must be done on rectified points,either by rectifying the input images (before step 302) or justrectifying the corresponding feature (before step 308). Rectificationparameters (i.e. stereo parameters) are then obtained (estimated) inoutput 206. In certain embodiments, when “disparity” is mentioned, it isassumed either rectified images or rectified image coordinates are used.In step 310 pixels with constant disparity over different time steps(while the vehicle was in motion) are labeled as “infinity” distance.The infinity offset e, defined as the disparity of points at infinity isthen estimated in step 312. In certain embodiments, this is done byaveraging the disparity of all infinity labeled pixels. In certainembodiments, just one infinity labeled pixel is enough, although inpractice a few dozen will be used.

Optionally, a step 314 of filtering stationary objects may be performedbefore estimating infinity offset e step 312. Objects that arestationary to the dual-camera and/or the given vehicle (e.g., anothervehicle moving with the same velocity and in the same direction as thegiven vehicle) will have constant disparity (same as infinity pixels)and therefore should be filtered from the infinity offset estimation.The filtering may include for example thresholding pixels with largeenough disparity (infinite disparity will be close to zero) or detectingcars/motorcycle/bikes by machine learning.

FIG. 3B shows a flowchart describing an exemplary embodiment of anothermethod for infinity correction. This embodiments is similar to the onein FIG. 3A, except for an added loop (iteration) step 316, whichiterates steps 302 to 308 to ensure that estimating infinity offset step312 has a sufficient number of infinity pixels (in general more infinitylabeled pixels are desired). i.e. reaches a steady state of infinityoffset estimation.

FIG. 4A shows a flowchart describing an exemplary embodiment of a methodfor estimating scale (estimating scaling factor s in equation 1′). As inthe estimation of e, the method is implemented using a dual-camera in adynamic environment (e.g. while driving in a given vehicle). At leastone set of stereo images is obtained in step 402 while in motion.Objects of known dimensions (OKDs) are detected in step 404 using adetection algorithms, by finding an OKD in one of the acquired images.We define a “detected OKD” as X_(OKD) X_(OKD) may be for example alicense plate length, a traffic speed sign diameter, or any otherobjects that are identical to each other and/or have constant dimensionsin a given place (city, state, country, continent, etc.). Thecorresponding points of each X_(OKD) are matched in the correspondingstereo image in step 406 and the size of known objects in pixels iscalculated in step 408. The size calculation may include using asegmentation algorithm to find all pixels associated with the object andto calculate its dimensions P_(OKD) (e.g. license plate length ortraffic speed sign diameter). The disparity of the known dimensionobject is calculated in step 410 using (as in step 308) rectified imagesor rectified image coordinates. The distance of the X_(OKD) from thedual-camera is calculated in step 412 using for example camera focallength and object pixel size as Distance=focal_length*X_(OKD)/P_(OKD).Scaling factor s is then estimated in step 414 using equation 1′ and thevalue of e from step 312.

In some embodiments, one set of images is needed since object dimensionsmay be known. In other embodiments, many sets of images can be obtained,preferably a thousand image sets, however fewer can be utilizedeffectively as well. A plurality of output estimations for s may beaverages over many measurements.

FIG. 4B shows a flowchart describing another exemplary embodiment of amethod for estimating scaling factor s. At least two sets of stereo (Land R) images are obtained while in motion in step 420. Stationaryobjects relative to the ground are found (detected) (e.g. by a detectionalgorithm for traffic sign/traffic lights, buildings, and/orcross-roads) in step 422. Corresponding points in each set are matchedin step 424 in a manner similar to that in step 304. Correspondingpoints between images in the two sets are matched in step 426 in amanner similar to that in step 306. A disparity map of correspondingpoints is generated in step 428 in a manner similar to that in step 308.A distance AZ driven by the vehicle between the taking of each pair ofsets of images is obtained (measured) in step 430, e.g. using thevehicle's velocity meter/GPS/external inertial measurement unit. Thedisparity of the stationary objects disparity and the driven distanceare then used to estimate scaling factor s in step 432, using equation1′ and equation 2 below (after e has been estimated in step 312).

$\begin{matrix}{{\Delta\; Z} = {{Z_{i + 1} - Z_{i}} = {\frac{s*f*B}{ps}\left( {\frac{1}{D_{i + 1} + e} - \frac{1}{D_{i} + e}} \right)}}} & (2)\end{matrix}$Furthermore, s can be easily extracted and averaged across many samples.

In an alternate embodiment, a dual-camera system obtains a set of twoimages from the dual-camera. The system performs local registration ofthe set of two images and obtains a registration map. The systemproceeds by finding the minimal disparity in the registration map,calculating the minimum of minimum disparity value, defining a globalminimum disparity value, and calibrating the dual-camera using theglobal minimum disparity.

Image registration is the process of transforming different sets of datainto one coordinate system. The data may be multiple photographs, datafrom different sensors, times, depths, or viewpoints.

FIG. 5 shows schematically an embodiment of an electronic devicenumbered 500 including a dual-aperture camera (as a particular exampleof a multi-aperture camera that can have more than two camera modules).Electronic device 500 comprises a first camera module 502 that includesa first lens module 504 that forms a first image recorded by a firstimage sensor 506 and a second camera module 510 that includes a secondlens module 512 that forms an image recorded by a second image sensor514. The two camera modules may be identical on different. For example,the two cameras may have similar or different FOVs. The cameras may beof different type, for example having image sensors sensitive to thevisible (VIS) wavelength range or to the infrared (IR) or otherwavelength range, time of flight (TOF) cameras, etc. Electronic device500 may further comprise a processing unit or application processor (AP)520. In some embodiments, initial or previous calibration data may bestored in memory 524 of the electronic device 500.

In use, a processing unit such as AP 520 may receive respective firstand second image data (or 1^(st) and 2^(nd) images) from camera modules502 and 510 and may supply camera control signals to camera modules 502and 510 to ensure that both images are acquired simultaneously. Afterreceiving at least one image from each camera, AP 520 will execute theprocesses described in FIGS. 2A, 2B, 3A, 3B and 4A, 4B. The finaloutcome will be updated stereo calibration parameters that may be storedin the memory unit 524, for further use.

It should be understood that where the claims or specification refer to“a” or “an” element, such reference is not to be construed as therebeing only one of that element.

Methods described herein can be implemented to calibrate cameraparameters as often as every time a user turns on a car or multipletimes per use or scheduled calibration periods preset by themanufacturer or user prompt, to a single calibration upon leaving thefactory, or a combination thereof. The present application does notrequire network or cloud access however can benefit from having suchaccess for storing or processing data, for example storage of images,accessing dimension data, remote processing, etc.

The disclosed embodiments are capable of processing sets of image pairsindependently, providing better results than the standard techniques.The disclosed methods can be done without a strict sequentialrequirement, unlike SFM, which requires a sequence of 20-100 imagepairs. Further, they are unique when compared with other known processesand solutions in that they (1) reduce the computational demand on thesystem, and (2) reduce the number of images needed to calibrate theparameters.

Unless otherwise defined, all technical or/and scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the application pertains.

While this disclosure describes a limited number of embodiments, it willbe appreciated that many variations, modifications and otherapplications of such embodiments may be made. In general, the disclosureis to be understood as not limited by the specific embodiments describedherein, but only by the scope of the appended claims.

What is claimed is:
 1. A method for stereo calibration of a dual-camerathat includes a first camera and a second camera, the method comprising:a) obtaining optimized extrinsic and intrinsic parameters using initialintrinsic parameters and, optionally, initial extrinsic parameters, byobtaining with the dual-camera a set of left and right images, matchingcorresponding points in the left and right images to obtain matchedfeature points, calculating extrinsic parameters, and refining theinitial intrinsic parameters based on the matched feature points; b)estimating an infinity offset e using the optimized extrinsic andextrinsic parameters; c) estimating a scaling factor s using theoptimized extrinsic and extrinsic parameters and infinity offset e; andd) using the optimized extrinsic and extrinsic parameters, infinityoffset e and scaling factor s to provide stereo calibration that leadsto improved depth estimation.
 2. The method of claim 1, wherein thestereo calibration includes dynamic stereo calibration.
 3. The method ofclaim 2, wherein the dynamic stereo calibration is performed in a movingvehicle that includes the dual-camera.
 4. The method of claim 1, whereinthe initial intrinsic parameters include nominal values of intrinsicparameters of the first and second cameras.
 5. The method of claim 1,wherein the initial intrinsic parameters include factory calibratedinitial intrinsic parameters of the first and second cameras.
 6. Themethod of claim 1, wherein the initial intrinsic parameters includeinitial intrinsic parameters of the first and second cameras estimatedindependently from bundle adjustment.
 7. The method of claim 1, whereinthe initial intrinsic parameters include initial intrinsic parameters ofthe first and second cameras estimated independently from structure frommotion.
 8. The method of claim 1, further comprising iterating betweenthe calculating extrinsic parameters and the matching correspondingpoints in the left and right images until sufficient matched points aregathered.
 9. The method of claim 1, wherein the refining the initialintrinsic parameters includes calculating a difference in image locationfor each matched feature point along a non-disparity axis and whereinthe obtaining optimized intrinsic parameters include fulfilling a stopcondition.
 10. The method of claim 8, wherein the refining the initialintrinsic parameters includes calculating a difference in image locationfor each matched feature point along a non-disparity axis and whereinthe obtaining optimized intrinsic parameters include fulfilling a stopcondition.
 11. A method for stereo calibration of a dual-camera thatincludes a first camera and a second camera, comprising: obtainingoptimized extrinsic and intrinsic parameters using initial intrinsicparameters and, optionally, initial extrinsic parameters by obtainingdynamically at least two sets of stereo images, wherein each stereoimage set includes a left image and a right image, matchingcorresponding points in left and right images of each set, matchingcorresponding points in, respectively, left images and right images ofat least two sets, generating a disparity map by calculating disparityvalues in the two time frames for all features matched in same sets andbetween sets, labeling pixels with constant disparity over differenttime steps as respective points at infinity and estimating infinityoffset e, from a respective disparity of the points at infinity,estimating an infinity offset e using the optimized extrinsic andextrinsic parameters; estimating a scaling factor s using the optimizedextrinsic and extrinsic parameters and infinity offset e; and using theoptimized extrinsic and extrinsic parameters, infinity offset e andscaling factor s to provide stereo calibration that leads to improveddepth estimation.
 12. A method for stereo calibration of a dual-camerathat includes a first camera and a second camera, the method comprising:obtaining optimized extrinsic and intrinsic parameters using initialintrinsic parameters and, optionally, initial extrinsic parameters,estimating an infinity offset e using the optimized extrinsic andextrinsic parameters; estimating a scaling factor s using the optimizedextrinsic and extrinsic parameters and infinity offset e by obtainingdynamically at least one set of stereo images, detecting in the set atleast one object of known dimensions (OKD) to obtain a detected OKDmarked X_(OKD), matching corresponding points in X_(OKD), calculating asize of X_(OKD), calculating a disparity of X_(OKD), calculating adistance of X_(OKD) from the dual-camera and estimating scaling factor susing the size, the disparity and the distance; and using the optimizedextrinsic and extrinsic parameters, infinity offset e and scaling factors to provide stereo calibration that leads to improved depth estimation.13. The method of claim 2, wherein the estimating scaling factor s usingthe optimized extrinsic and extrinsic parameters and infinity offset eincludes obtaining dynamically at least one set of stereo images,detecting in the set at least one object of known dimensions (OKD) toobtain a detected OKD marked X_(OKD), matching corresponding points inX_(OKD), calculating a size of X_(OKD), calculating a disparity ofX_(OKD), calculating a distance of X_(OKD) from the dual-camera andestimating scaling factor s using the size, the disparity and thedistance.
 14. The method of claim 3, wherein the estimating scalingfactor s using the optimized extrinsic and extrinsic parameters andinfinity offset e includes obtaining dynamically at least one set ofstereo images, detecting in the set at least one object of knowndimensions (OKD) to obtain a detected OKD marked X_(OKD), matchingcorresponding points in X_(OKD), calculating a size of X_(OKD),calculating a disparity of X_(OKD), calculating a distance of X_(OKD)from the dual-camera and estimating scaling factor s using the size, thedisparity and the distance.
 15. The method of claim 11, wherein theestimating scaling factor s using the optimized extrinsic and extrinsicparameters and infinity offset e includes obtaining dynamically at leastone set of stereo images, detecting in the set at least one object ofknown dimensions (OKD) to obtain a detected OKD marked X_(OKD), matchingcorresponding points in X_(OKD), calculating a size of X_(OKD),calculating a disparity of X_(OKD), calculating a distance of X_(OKD)from the dual-camera and estimating scaling factor s using the size, thedisparity and the distance.
 16. A method for stereo calibration of adual-camera that includes a first camera and a second camera, the methodcomprising: obtaining optimized extrinsic and intrinsic parameters usinginitial intrinsic parameters and, optionally, initial extrinsicparameters, estimating an infinity offset e using the optimizedextrinsic and extrinsic parameters; estimating a scaling factor s usingthe optimized extrinsic and extrinsic parameters and infinity offset eby estimating scaling factor s using the optimized extrinsic andextrinsic parameters and infinity offset e includes obtainingdynamically at least two sets of stereo images, detecting in the sets atleast one stationary object X_(OS), matching corresponding points inX_(OS) to obtain a disparity, obtaining a distance driven between theobtaining of the at least two sets, and estimating scaling factor susing the disparity and the distance; and using the optimized extrinsicand extrinsic parameters, infinity offset e and scaling factor s toprovide stereo calibration that leads to improved depth estimation. 17.The method of claim 2, wherein the estimating scaling factor s using theoptimized extrinsic and extrinsic parameters and infinity offset eincludes obtaining dynamically at least two sets of stereo images,detecting in the sets at least one stationary object X_(OS), matchingcorresponding points in X_(OS) to obtain a disparity, obtaining adistance driven between the obtaining of the at least two sets, andestimating scaling factor s using the disparity and the distance. 18.The method of claim 3, wherein the estimating scaling factor s using theoptimized extrinsic and extrinsic parameters and infinity offset eincludes obtaining dynamically at least two sets of stereo images,detecting in the sets at least one stationary object X_(OS), matchingcorresponding points in X_(OS) to obtain a disparity, obtaining adistance driven between the obtaining of the at least two sets, andestimating scaling factor s using the disparity and the distance. 19.The method of claim 11, wherein the estimating scaling factor s usingthe optimized extrinsic and extrinsic parameters and infinity offset eincludes obtaining dynamically at least two sets of stereo images,detecting in the sets at least one stationary object X_(OS), matchingcorresponding points in X_(OS) to obtain a disparity, obtaining adistance driven between the obtaining of the at least two sets, andestimating scaling factor s using the disparity and the distance.